Conventionally, technologies such as Simple Object Access Protocol (SOAP) with Attachments (SwA) and File Transfer Protocol (FTP) based applications are used to transport large XML files between systems. Also, parsers such as Streaming Application Programming Interface for XML (StAX) or Document Object Model (DOM) parsers are used to parse the transported XML file. In order to maintain a reasonable performance, the transportation must be coupled with a caching mechanism, as, such large XML files cannot be held in Random Access Memory (RAM) due to its limited capacity in various computing systems. Further, it is also infeasible to parse a large portion of the XML file every time a record or a set of records are needed.
Most of the existing solutions for transporting large XML files between systems are not generic and hence end-users (developers) are not able to re-use code pertaining to the existing solutions. The existing solutions are thus associated with a steep learning curve. This limitation of existing solutions forces a developer to invest valuable time in learning to use a new framework or Application Programming Interface (API) rather than concentrating on implementing business logic.
Consequently, there is need for a generic system and method for transporting large XML files between systems/networks efficiently. There is need for a generic transportation system and method which would allow end-users to fully concentrate on developing business logic and which would completely abstract the end-users from scalability issues, such as, using special data structures to handle extremely large XML files, and the operation of the transportation system itself. Further, there is need for a system and method for transportation of large XML files which is memory efficient and employs an effective caching mechanism.